Add importable Datadog dashboard example - #142
Merged
Merged
Conversation
Completes the dashboard set (Grafana + Tempo, SigNoz, now Datadog) so a team already running Datadog can visualize governed runs from the same OpenTelemetry GenAI spans RiskKernel exports — no extra instrumentation. The dashboard JSON is in Datadog's import schema (ordered layout, data_source: spans) and reaches parity with the Grafana/SigNoz panels: total/over-time spend, spend by run, halts by reason, tool outcomes, latency and token burn by model, model-call rate. Every query references the attribute names pinned in api/v1/otel-genai.md, with Datadog's @-prefix for span attributes (@riskkernel.cost.usd, @gen_ai.request.model, ...) and the built-in @duration measure for latency. The README covers pointing the OTLP exporter at the Datadog Agent's OTLP receiver (gRPC 4317 / HTTP 4318) — the Datadog API key sits on the Agent, not RiskKernel, which only emits OTLP — plus the @-prefix gotcha, the service-name template variable, retention/indexed-span behavior, and the optional span-based-metric path for durable long-window cost rollups. Cross-linked from the Grafana/SigNoz/main observability READMEs.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Completes the OTel dashboard set — Grafana + Tempo and SigNoz already shipped; this adds the Datadog import so a team already running Datadog can visualize governed runs straight from the OpenTelemetry GenAI spans RiskKernel exports. No extra instrumentation: Datadog ingests the spans over OTLP, and you import one JSON file.
What's here
examples/otel/datadog/dashboards/riskkernel-agent-runs.json— Datadog's importable dashboard schema (layout_type: ordered,data_source: spans). Panel parity with Grafana/SigNoz:examples/otel/datadog/README.md— how to point the OTLP exporter at the Datadog Agent's OTLP receiver (gRPC4317/ HTTP4318), import the dashboard, and what each panel queries.Datadog specifics (called out honestly in the README)
DD_API_KEYis standard Agent config.@prefix (@riskkernel.cost.usd,@gen_ai.request.model); tags have none. Datadog's OTLP intake preserves the dotted names, so they read identically to the pinned set.@durationis Datadog's intrinsic span-duration measure (nanoseconds) for the latency panel — intrinsic, not a RiskKernel attribute.data_source: spanspanels query retained spans; the README notes keeping a retention filter for RiskKernel's spans, and documents the optional span-based metric upgrade for durable, long-window cost rollups (and the billing note).servicetag; the dashboard ships a$servicetemplate variable defaulting toriskkernel.Contract fidelity
Every attribute the queries touch matches
api/v1/otel-genai.mdexactly (verified programmatically):gen_ai.operation.name,gen_ai.request.model,gen_ai.usage.output_tokens,riskkernel.cost.usd,riskkernel.run.id,riskkernel.tool.status,riskkernel.halt.reason— plus the built-in@duration. The JSON parses cleanly. Cross-linked from the Grafana, SigNoz, and main observability READMEs, and a CHANGELOG entry under[Unreleased].Closes #89